Camera calibration typically relates to the process of estimating, or calculating, parameters associated with a camera that may affect image processing. These parameters may be referred to as calibration parameters. The calibration parameters may include parameters of the lens and image sensor of the camera, and/or parameters relating to the physical location and/or orientation of the camera relative to a scene. Knowledge of these parameters may be used for various aspects of the imaging process. For example, knowledge of the calibration parameters can be used to correct for lens distortion, measuring the size of an object in world coordinates and determining the location and/or orientation of the camera relative to a scene. As such, knowledge of a set of camera calibration parameters may be useful in a variety of applications including, for example, navigation systems, augmented reality scene construction, 3-D scene reconstruction or robotics.
Calibration parameters may broadly be classified into two groups: extrinsic parameters and intrinsic parameters. Extrinsic parameters define the location and orientation of the camera with respect to a scene. More particularly, the extrinsic parameters may define the location and orientation of a camera reference frame with respect to a world reference frame. That is, the extrinsic parameters may map camera coordinates to world coordinates. The origin of the camera coordinate system, or reference frame, may conveniently be taken to be at the camera's optical centre. The extrinsic parameters may comprise translational and rotational components.
The intrinsic parameters may map the camera coordinates into the image plane of the camera. The intrinsic parameters may represent a transformation from the camera's 3D coordinates into 2D image coordinates. The intrinsic parameters may additionally map coordinates in the image plane to pixel coordinates of an image. Thus the intrinsic parameters may map coordinates in the camera reference frame to pixel coordinates of an image point. The intrinsic parameters may comprise one or more of the camera's focal length, optical centre (also known as the principle point) and pixel skew coefficients.
A schematic illustration of the relationship between the intrinsic and extrinsic parameters is shown in FIG. 1.
A triangular object 101 is an object of a scene having a position in the scene specified with respect to a three-dimensional world coordinate system, or world frame, (Xw, YW, ZW). A vertex of the triangle 105 is a scene point Pw having a position in the world coordinate frame given by Pw=(xw, yw, zw).
A camera (not shown) defines a three-dimensional camera coordinate system, or camera frame, (Xc,Yc, Zc). The origin of the camera frame is the optical centre, or centre of perspective projection of the camera. The camera has an associated image plane 103 located at a distance f from the optical centre, where f is the focal length of the camera. Points within the image plane can be defined with respect to a two-dimensional image plane coordinate system (or image plane frame), denoted (Xi, Yi). The origin of the image plane frame may be coincident with the centroid of the image plane.
The scene point Pw corresponds to an associated image point Pi having a position in the image plane frame given by Pi=(xi, yi).
The extrinsic parameters may be those that define the transformation between the known world reference frame (Xw, YW, ZW) and the unknown camera frame (Xc, Yc, Zc). The position of the scene point in the camera reference frame is denoted PC. Thus, in this example, the extrinsic parameters may define the transformation between the position of the scene point (Pw=(xw, yw, zw)) in the world reference frame and its position in the camera frame (Pc=(xc, yc, zc)). The intrinsic parameters may define the transformation between the camera reference frame and the image plane reference frame (i.e. the mapping between the position of a point defined in the camera reference frame (Pc=(xc, yc, zc)) and its associated position in the image plane frame (Pi=(xi, yi)). Thus it can be appreciated that knowledge of both the necessary extrinsic and intrinsic parameters can define a mapping, or transformation, between points Pw and Pi.
The intrinsic parameters may further define a transformation between image plane coordinates and pixel coordinates. The pixel coordinates may be defined with respect to a pixel frame that lies in the image plane. The transformation between the image plane reference frame and pixel frame may account for artefacts such as lens distortion and pixel skew.
One way to obtain values for the calibration parameters is to perform a calibration using a known calibration pattern. In this approach one or more images are taken of a calibration pattern located in a known position in space and having a known geometry. The calibration parameters are then estimated using these images and the known position and geometry of the calibration pattern. This approach suffers from the drawback that it requires the use of a dedicated physical component (i.e. the calibration pattern). In addition, the camera cannot be used for its intended purpose during the calibration procedure because it is restricted to imaging the calibration pattern.
An alternative approach to obtain values of the calibration parameters is to estimate them from ‘real’ image sequences captured by the camera (i.e. image sequences captured by the camera during its intended use). In this regard, algorithms have been developed for use in performing real-time, or per-frame, camera calibration. An example of one such algorithm is the Simultaneous Localization and Mapping (SLAM) algorithm.